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Executive Summary 

Customer: NASA’s Launch Services Program (LSP), Ground Systems Development and 
Operations (GSDO), and Space Launch System (SLS) programs 

NASA’s LSP, GSDO, SLS and other programs at Kennedy Space Center (KSC) and Cape 
Canaveral Air Force Station (CCAFS) use the daily and weekly weather forecasts issued by the 
45th Weather Squadron (45 WS) as decision tools for their day-to-day and launch operations on 
the Eastern Range (ER). For example, to determine if they need to limit activities such as 
vehicle transport to the launch pad, protect people, structures or exposed launch vehicles given 
a threat of severe weather, or reschedule other critical operations. The 45 WS uses numerical 
weather prediction models as a guide for these weather forecasts, particularly the Air Force 
Weather Agency (AFWA) 1 .67 km Weather Research and Forecasting (WRF) model. 

Considering the 45 WS forecasters’ and Launch Weather Officers’ (LWO) extensive use of 
the AFWA model, the 45 WS proposed a task at the September 2013 Applied Meteorology Unit 
(AMU) Tasking Meeting requesting the AMU verify this model. Due to the lack of archived model 
data available from AFWA, verification is not yet possible. Instead, the AMU proposed to 
implement and verify the performance of an ER version of the AMU high-resolution WRF 
Environmental Modeling System (EMS) model (Watson 2013) in real-time. The tasking group 
agreed to this proposal; therefore the AMU implemented the WRF-EMS model on the second of 
two NASA AMU modeling clusters. 

The model was set up with a triple-nested grid configuration over KSC/CCAFS based on 
previous AMU work (Watson 2013). The outer domain (D01) has 12-km grid spacing, the middle 
domain (D02) has 4-km grid spacing, and the inner domain (D03) has 1.33-km grid spacing. 
The model runs a 12-hr forecast every hour, D01 and D02 domain outputs are available once 
an hour and D03 is every 15 minutes during the forecast period. 

The AMU assessed the WRF-EMS 1.33-km domain model performance for the 2014 warm 
season (May-September). Verification statistics were computed using the Model Evaluation 
Tools, which compared the model forecasts to observations. The mean error values were close 
to 0 and the root mean square error values were less than 1 .8 for mean sea-level pressure 
(mb), temperature (K), dewpoint temperature (K), and wind speed (ms 1 ), all very small 
differences between the forecast and observations considering the normal magnitudes of the 
parameters. The precipitation forecast verification results showed consistent under-forecasting 
of the precipitation object size. This could be an artifact of calculating the statistics for each hour 
rather than for the entire 12-hour period. The AMU will continue to generate verification statistics 
for the 1 .33-km WRF-EMS domain as data become available in future cool and warm seasons. 
More data will produce more robust statistics and reveal a more accurate assessment of model 
performance. 

Once the formal task was complete, the AMU conducted additional work to better 
understand the wind direction results. The results were stratified diurnally and by wind speed to 
determine what effects the stratifications would have on the model wind direction verification 
statistics. The results are summarized in the addendum at the end of this report. 

In addition to verifying the model’s performance, the AMU also made the output available in 
the Advanced Weather Interactive Processing System II (AWIPS II). This allows the 45 WS and 
AMU staff to customize the model output display on the AMU and Range Weather Operations 
AWIPS II client computers and conduct real-time subjective analyses. In the future, the AMU will 
implement an updated version of the WRF-EMS model that incorporates local data assimilation. 
This model will also run in real-time and be made available in AWIPS II. 
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1 Introduction 


NASA’s Launch Services Program, Ground Systems Development and Operations, Space 
Launch System and other programs at Kennedy Space Center (KSC) and Cape Canaveral Air 
Force Station (CCAFS) use the daily and weekly weather forecasts issued by the 45th Weather 
Squadron (45 WS) as decision tools for their day-to-day and launch operations on the Eastern 
Range (ER). Examples include determining if they need to limit activities such as vehicle 
transport to the launch pad, protect people, structures or exposed launch vehicles given a threat 
of severe weather, or reschedule other critical operations. The 45 WS uses numerical weather 
prediction models as a guide for these weather forecasts, particularly the Air Force Weather 
Agency (AFWA) 1 .67 km Weather Research and Forecasting (WRF) model. 

Considering the 45 WS forecasters’ and Launch Weather Officers’ (LWO) extensive use of 
the AFWA model, the 45 WS proposed a task at the September 2013 Applied Meteorology Unit 
(AMU) Tasking Meeting requesting the AMU verify this model. Due to the lack of archived model 
data available from AFWA, verification is not yet possible. Instead, the AMU proposed to 
implement and verify the performance of an ER version of the high-resolution WRF 
Environmental Modeling System (EMS) model configured by the AMU (Watson 2013) in real 
time. Implementing a real-time version of the ER WRF-EMS would generate a larger database 
of model output than in the previous AMU task for determining model performance, and allows 
the AMU more control over and access to the model output archive. 

The tasking group agreed to this proposal; therefore the AMU implemented the WRF-EMS 
model on the second of two NASA AMU modeling clusters. The AMU also calculated verification 
statistics to determine model performance compared to observational data. Finally, the AMU 
made the model output available on the AMU Advanced Weather Interactive Processing System 
II (AWIPS II) servers, which allows the 45 WS and AMU staff to customize the model output 
display on the AMU and Range Weather Operations (RWO) AWIPS II client computers and 
conduct real-time subjective analyses. 



2 Model Installation and Configuration 

The first part of this task was to set up the model to run in real time. Once the NASA AMU 
modeling cluster was configured for AMU use, the AMU installed and configured WRF-EMS, 
and completed a few test runs. After confirming WRF-EMS was running properly, the AMU set 
up a triple-nested grid configuration over KSC/CCAFS based on a previous AMU task (Watson 
2013). The results of that work showed the best configuration for the ER used the Advanced 
Research WRF core with the Lin microphysical scheme and the Yonsei University planetary 
boundary layer scheme. Once the model setup was configured and tested, the AMU automated 
WRF-EMS to run a 12-hr forecast every hour. The model is initialized every hour using the 
National Centers for Environmental Prediction’s (NCEP) 13-km Rapid Refresh model for 
boundary and initial conditions, Short-term Prediction Research and Transition Center (SPoRT) 
Land Information System land surface data, and SPoRT sea surface temperature data. 

Figure 1 shows the boundaries of the three domains included in this task. D01 is the outer 
domain with 12-km grid spacing, D02 is the middle domain with 4-km grid spacing, and D03 is 
the inner domain with 1.33-km grid spacing. D01 and D02 domain output is available once an 
hour and D03 is available every 15 minutes during the 12-hr forecast period. The boundaries of 
D01 , which include much of the eastern United States, were selected should this work grow to 
include Wallops Flight Facility in the future. D03 is centered over the ER and is the domain for 
which the AMU calculated verification statistics to determine model performance. 
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Figure 1 . Map of the eastern United States showing the boundaries of 
each domain. The outer domain (cyan rectangle, D01) has 12-km grid 
spacing, the middle domain (green rectangle, D02) has 4-km grid spacing, 
and the inner domain (yellow rectangle, D03) has 1 .33-km grid spacing. 
The AMU calculated verification statistics for the inner domain, D03. 
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3 Model Forecast Verification 


The AMU assessed model performance by computing statistics that compared the model 
forecasts to observations. The mean error (ME) and root mean square error (RMSE) were 
calculated for the surface parameter forecasts, and precipitation forecasts were compared to 
nationally available rainfall data using a technique developed at the National Center for 
Atmospheric Research (NCAR). 

Observational Data 

In order to determine the WRF-EMS model performance, the AMU required surface weather 
observations of temperature (T), dewpoint temperature (T d ), wind speed and direction, and 
mean sea-level pressure (MSLP) as well as rainfall data. Based on the previous AMU model 
verification work (Watson 2013), the AMU used the NCEP Meteorological Assimilation Data 
Ingest System (MADIS) and Stage IV precipitation data for the observational datasets. 

3.1.1 MADIS 

The AMU coordinated with NCEP and set up a data connection to the NASA AMU cluster. 
With this connection the AMU automatically receives MADIS observational data using the local 
data manager software in real time. MADIS includes multiple data types including METAR and 
mesonet files. METAR is the international standard code format for hourly surface weather 
observations. Mesonet refers to a network of automated weather stations designed to observe 
mesoscale meteorological phenomena and report conditions in time intervals anywhere from 1 
to 15 minutes. These data were used to verify hourly 2-m T (K), T d (K), 10-m wind speed (ms 1 ) 
and direction (degrees), and MSLP (mb). Figure 2 shows the locations of the METAR and 
mesonet weather stations (https://madis-data.noaa.gov/sfc display/) that were used to verify the 
performance of the WRF-EMS inner domain. 
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(blue squares with red circles) weather station locations from 
MADIS. The AMU used the data from these sites to calculate 
verification statistics for the inner WRF-EMS domain. 

3.1.2 Stage IV 

In addition to the MADIS data, the AMU used gridded NCEP Stage IV precipitation data 
(http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage4/) to verify the hourly WRF-EMS 
precipitation forecasts. The Stage IV data combines radar data and rain gauge reports to 
produce hourly rainfall accumulation on a 4-km grid. It is a manually quality-controlled mosaic 
from the regional 1 -hr precipitation analyses produced by 12 National Weather Service Forecast 
Centers (Lin and Mitchell 2005). 

Verification Software 

The AMU calculated verification statistics to determine the WRF-EMS model performance 
using the Model Evaluation Tools (MET) software. MET was developed by the NCAR 
Developmental Testbed Center through the support of AFWA and the National Oceanic and 
Atmospheric Administration. It was designed to be a highly configurable, state-of-the-art suite of 
verification tools (MET User’s Guide 2013). The AMU used two of the statistical verification tools 
available in MET for this task: the Point-Stat tool and the Method for Object-Based Diagnostic 
Evaluation (MODE) tool. 
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3.1.3 Point-Stat Tool 


The Point-Stat tool was used to compute traditional verification scores for the hourly surface 
forecasts including 2-m T (K), T d (K), 10-m wind speed (ms 1 ) and direction (degree), and MSLP 
(mb). It compares the WRF-EMS forecast to the corresponding MADIS point observations. This 
tool was run on each hourly forecast and consolidated to determine the overall model 
performance. 

3.1.4 MODE Tool 

The MODE tool applies an object-based verification technique in comparing a gridded 
forecast to a gridded analysis. MODE was used to compare the WRF-EMS precipitation 
accumulation forecasts to the NCEP Stage IV observations. 

In order to use MODE for verification, the timing and grid spacing of the forecast must match 
observational data. Since the WRF-EMS produces precipitation forecasts every 15 minutes on a 
1.33-km grid and the Stage IV data is available hourly on a 4-km grid, the AMU had to reformat 
the WRF-EMS output to match the observations. Each of the 15-minute WRF-EMS files were 
converted from 1.33-km to a 4-km grid and then combined to create a 1 -hr 4-km WRF-EMS 
precipitation accumulation forecast file. The MODE tool ingested these new files and compared 
them to the Stage IV data for verification. This process was repeated for every forecast hour. 

Warm Season Verification Results 

The AMU calculated verification statistics to determine the WRF-EMS 1 .33-km domain 
model performance for the 2014 warm season (May-September) using the tools described in 
section 0. 

3.1.5 Surface Parameters 

The Point-Stat tool was used to compute the ME and RMSE for the hourly 2-m MSLP, T, T d , 
and 10-m wind speed and direction. The ME is the overall bias of the model parameter during 
the period of interest, calculated by subtracting the observation from the forecast and averaging 
the differences. It determines whether there is a positive, negative, or no bias in the model 
forecast for any parameter. ME values range from negative infinity to infinity with a perfect score 
equal to 0. 


ME = 



- o, 


) 


where: 

n = number of model output times and/or vertical levels over the forecast period, 
fi = WRF forecast of MSLP, T, T d , wind speed, or wind direction, and 
o, = observed MSLP, T, T d , wind speed, or wind direction. 

The RMSE isolates the magnitude of the model error as ME can mask the variance in the 
differences between the forecast and observations. An ME close to 0 could be the result of 
averaging larger negative and positive differences. The RMSE equation squares the 
differences, calculates an average of the squared differences, then calculates the square root of 
the average value. As with ME, smaller RMSE values indicate better model performance. It 
ranges from 0 to infinity with a perfect score equal to 0. 
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RMSE = F^C/i-Oi) 2 

^ n i=i 

where n, /j, and o, are defined as above. 

Figure 3 shows the ME for MSLP, T, T d , and wind speed versus model forecast hour for the 
2014 warm season. The ME curve for T is close to 0 at all forecast hours, indicating a good 
forecast for this parameter. The ME values for wind speed and T d are within ±0.4 with wind 
speed showing a small positive bias and T d showing a slightly negative, or dry, bias. The values 
for MSLP are all negative ranging from -1 .2 to -1 .4. Considering the magnitude of MSLP values 
on the order of 1000 mb, these biases are very small in comparison. Figure 4 is the same but 
for wind direction. These values increase with forecast hour from just above 25 to 35 degrees. 
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Figure 3. The ME for each parameter versus forecast hour. Surface pressure is 
in mb (blue dots), temperature in K (green dots), dewpoint temperature in K (red 
dots) and wind speed in ms" 1 (purple dots). 
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Figure 4. The ME for wind direction in degrees versus forecast hour. 


Figures Figure 5 and Figure 6 are the same as Figures Figure 3 and Figure 4 respectively, 
but for RMSE. The RMSE values for MSLP, T, T d and wind speed slightly increase with forecast 
hour but remain between 1 and 1.8 (Figure 5). These values are close to 0, which confirms the 
magnitude of the model error for these parameters is very small. A similar trend exists for wind 
direction but the RMSE values range from about 40 to 50 degrees. These values could be 
because of light and variable winds that are common at night over KSC/CCAFS during the 
warm season. When winds are light there tends to be a large variance in the wind direction that 
would be difficult for the model to forecast. In the future, the AMU will stratify the warm season 
results diurnally to determine how the time of day influences the model wind direction forecasts. 
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Figure 5. Same as Figure 3 but for RMSE. 
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Figure 6. Same as Figure 4 but for RMSE. 

3.1.6 Precipitation 

The MODE tool was used to verify the model precipitation forecasts. This tool applies an 
object-based verification technique to compare a gridded forecast to a gridded analysis. The 
technique for defining objects in MODE is illustrated in Figure 7. Figure 7a is an example of raw 
gridded data. MODE uses two processes to convert raw gridded precipitation values into 
precipitation objects. The first step is to smooth the data (Figure 7b) and the second is to create 
a convolved field (Figure 7c). Once these steps are complete, MODE has defined final 
precipitation objects (Figure 7d) that are used in the verification statistics. A more detailed 
explanation about this process and MODE is in the MET User’s Guide, chapter 8 (2013). Table 
1 shows the statistics the AMU used from MODE for the model verification. 
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(a) 




Figure 7. Illustration of the technique used in 
the MODE tool to define precipitation objects: a) 
raw gridded precipitation data, b) smoothed 
data, c) convolved field, d) final field of objects 
used in verification statistics (from the MET 
users guide, Figure 8-1). 


Table 1 . List of statistics available in the MODE tool the AMU used to verify the model. 

Statistic Name 

Description 

Centroid Distance 

Distance between two object centroids (in grid units) 

Area Ratio 

Ratio of the areas of two objects defined as the lesser of the 
forecast area divided by the observation area or its reciprocal 
(unitless) 

Interest 

Total interest value computed for a pair of simple objects (unitless) 
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The centroid distance is the distance between the centers of two objects: the observed 
precipitation object and the corresponding forecast precipitation object. A perfect forecast would 
have a centroid distance equal to 0. Figure 8 shows the centroid distance versus model forecast 
hour for the warm season verification. As expected, there is a general increase in distance with 
time although it remains between 35 and 38 grid boxes. The distance in km is the centroid 
distance in grid boxes multiplied by the domain resolution of 1 .33-km, resulting in distances 
between 46.55 and 50.54 km. These location differences may be present because warm 
season convection remains one of the most poorly forecast meteorological parameters, in part 
due to dynamic and thermodynamic features that occur on the mesoscale (Watson 2007). Note 
that MODE does not calculate statistics for the initialization time (Forecast Hour = 0) since the 
model takes time to spin-up the precipitation forecasts. 
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Figure 8. Centroid distance versus model forecast hour for the warm season 
model verification. Centroid distances are in number of grid boxes. 


The area ratio compares the size of the forecast objects to the observation objects (forecast 
area divided by observation area). A perfect forecast would have an area ratio equal to 1. A 
value greater than 1 indicates the model is forecasting larger objects than observed, and a 
value less than one indicates the model is forecasting smaller objects than observed. Figure 9 
shows the area ratio versus model forecast hour remaining steady at about 0.35. This means 
the model is consistently under-forecasting the size of the precipitation objects. 
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Figure 9. Area ratio versus model forecast hour for the warm season model 
verification. 

Finally, the interest value compares the differences in attributes between the forecast and 
observed objects, including the centroid distance and area ratio, and gives an indication of the 
overall quality of the model precipitation forecasts. It ranges from 0 to 1 with a perfect score 
equal to 1. More detail about these statistics can be found in Brown et al. 2007. Figure 10 
shows the interest versus model forecast hour. These values consistently remain just above 0.6 
regardless of model forecast hour. 


1.00 

0.90 

0.80 

0.70 

~ 0.60 
0) 

5 0.50 
- 0.40 
0.30 
0.20 
0.10 
0.00 


Interest 

May - September 2014 




5 6 7 

Forecast Hour 


10 11 12 


Figure 10. Interest value versus model forecast hour for the warm season 
model verification. 
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3.1 .7 Summary of Results 

The ME values were close to 0 and the RMSE values were less than 1.8 for MSLP, T, T d , 
and wind speed, all very small differences between the forecast and observations considering 
the normal magnitudes of the parameters. The RMSE for wind direction was between 40 and 50 
degrees, which could be because of the light and variable winds that are common at night over 
KSC/CCAFS during the warm season. After the formal task was complete, the AMU stratified 
the data diurnally and by wind speed to determine how the wind direction verification statistics 
would be affected. The addendum at the end of this report provides the results of the updated 
wind speed and wind direction analyses. The precipitation forecast verification results showed 
consistent under-forecasting of the precipitation object size. This could be an artifact of 
calculating the statistics for each hour rather than for the entire 12-hour period. For example, if 
the timing of the model-forecast precipitation occurs within the hour before or after the observed 
precipitation, then the forecast will not match the observation. The model may also correctly 
forecast the location of the precipitation but the statistics could still show poor model 
performance due to timing. The AMU will continue to generate verification statistics for the 1 .33- 
km WRF-EMS domain as data become available in future cool and warm seasons. More data 
will produce more robust statistics and reveal a more accurate assessment of model 
performance. 
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4 WRF-EMS Output into AWIPS II 

In addition to verifying the model’s performance, the AMU also made the output available in 
AWIPS II. This allows the 45 WS and AMU staff to customize the model output display on the 
AMU and RWO AWIPS II client computers and conduct real-time subjective analyses. The AMU 
was able to customize the WRF-EMS runs and ingest the three WRF-EMS domains separately 
in order to display them individually in AWIPS II via the Common AWIPS Visualization 
Environment (CAVE). 

CAVE Examples 

Example CAVE screen shots of the 12-, 4- and 1.33-km AMU WRF-EMS model 
frontogenesis forecast product are shown in Figures Figure 11 , Figure 12 and Figure 13 , 
respectively. The AMU wrote Perl scripts that automate the ingest process and update the 
model in AWIPS II every hour. 



Figure 11. CAVE screen shot of the AMU WRF-EMS 12-km frontogenesis output valid at 
1900 UTC on 10 July 2014. The warm colors are frontogenesis and the cool colors are 
frontolysis. 
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Procedures to Display WRF-EMS Output 

Like all other model data available in AWIPS II, the AMU WRF-EMS is accessed via the 
CAVE Volume Browser. To open the Volume Browser, the user must select Volume and then 
Browser in the CAVE window (Figure 14). This will open the Volume Browser window. To 
access the AMU WRF-EMS forecasts, click the Volume drop-down list under Sources as 
indicated in Figure 15. A partial list of the model forecasts available via the Volume drop-down 
menu is shown in Figure 16. The three AMU WRF-EMS domains are at the top of the list. The 
user must then select the desired Field and Plane, for example Wind and Surface respectively, 
which will list the available products under Product Selection List (Figure 17). The product is 
loaded by either double clicking the item in the Product Selection List, or clicking the Load 
button. 



Figure 14. CAVE window highlighting how the user selects the Volume Browser to display 
model data. 
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Volume Browser 
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File Edit Tools Plan view Time 


^Sources^ Fields 

Volume ^SfcGrid - Point * Basic - Wind - Derived * Sfc/2D - Other Radar Ensemble 


Planes 

Pres - Theta - Hgt - Temp - Tilts - Misc - Lyrs 


Product Selection List 


Inventory 


Selected for loading: 0 


Figure 15. Volume Browser window in CAVE. The AMU WRF-EMS is selected by clicking the 
Volume drop-down menu under Sources. 
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Figure 16. A partial 
list of available 
models from the 
Volume drop-down 
showing the three 


AMU WRF-EMS 
domains at the top 
of the list. 
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Figure 17. Volume Browser showing the selection of AMU WRF-EMS parameters and 
associated available products. 

Figure 18 shows the products in the main CAVE window based on the selections highlighted 
in Figure 17. The surface wind barbs within the 12-km, outer-domain are shown in green. The 
bottom right corner of the window shows the loaded product details. Figure 19 is a zoomed in 
image of the details. The red box outlines the product name, in this example it is the AMU WRF- 
EMS 12-km surface winds. Next, the blue box shows the date and time (Z) the model was run. 
The third box (purple) is the forecast hour (0-12 hours). Finally, the green box shows the 
product’s valid date and time. 



Figure 18. AMU WRF-EMS 12-km forecast surface wind barbs (green). 
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Figure 19. Zoomed in image of the product details shown in Figure 18. The red box outlines 
the product name, the blue box is the date and model run time (Z), the purple box is the 
forecast hour and the green box is the valid date and time. 

5 Summary and Future Work 

Given the extensive use of the AFWA 1.67 WRF model in operations, the 45 WS proposed 
a task at the September 2013 AMU Tasking Meeting requesting the AMU verify this model. 
Since there is a lack of archived model data available from AFWA, verification is not yet 
possible. Therefore, the AMU proposed to implement and verify the performance of an ER 
version of the AMU high-resolution WRF-EMS model in real-time. The tasking group agreed to 
this proposal and therefore the AMU implemented the WRF-EMS model on the second of two 
NASA AMU modeling clusters to begin producing model output for the verification. The model 
was set up with a triple-nested grid configuration over KSC/CCAFS based on a previous AMU 
task (Watson 2013). 

The AMU assessed the WRF-EMS 1.33-km model performance for the 2014 warm season. 
Verification statistics were computed using the MET software, which compared the model 
forecasts to observations. The observational datasets included MADIS surface observations 
and Stage IV gridded precipitation data from NCEP. The ME values were close to 0 and the 
RMSE values were less than 1 .8 for MSLP, T, T d , and wind speed, all very small differences 
between the forecast and observations considering the normal magnitudes of the parameters. 
The RMSE for wind direction was between 40 and 50 degrees, which could be because of the 
light and variable winds that are common at night over KSC/CCAFS during the warm season. 
After the formal task was complete, the AMU conducted additional work to better understand the 
results found for wind direction. This report’s addendum provides the results of the updated 
wind speed and wind direction analyses. The precipitation forecast verification results showed 
consistent under-forecasting of the precipitation object size. This could be an artifact of 
calculating the statistics for each hour rather than for the entire 12-hour period. The AMU will 
continue to generate verification statistics for the 1 .33-km WRF-EMS domain as data become 
available in future cool and warm seasons. As a potential future task, the AMU could conduct a 
more in-depth literature review and recalculate verification statistics once a larger database has 
been acquired. More data will produce more robust statistics and reveal a more accurate 
assessment of model performance. 

In addition to the verification, the AMU also made the output available in AWIPS II. This 
allows the 45 WS and AMU staff to customize the model output display on the AMU and RWO 
AWIPS II client computers and conduct real-time subjective analyses. The LWOs and Range 
Weather Forecasters have found the AMU-WRF model's performance quite useful over the 
entire summer months during 2014. It has frequently been the preferred model to help 
accurately identify complex, very small scale boundary interactions. Of particular note, its rapid 
hourly update capability allows for constant correction and fine-tuning, which further aids the 
LWOs during sensitive ground and launch operations that require specific timing accuracy. In 
the future, the AMU will implement an updated version of the WRF-EMS model that 
incorporates local data assimilation based on a recently completed AMU task (Watson 
201 4).This model will also run in real-time and be made available in AWIPS II. 
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Addendum 


Once the formal task was complete, the AMU conducted additional work to better 
understand the results found for wind direction (see Figure 4 and Figure 6). Initially, the AMU 
stratified the results diurnally to determine how the daytime/nighttime periods influenced the 
model wind direction. The results showed little difference between the daytime and nighttime 
values. Given these findings, the AMU stratified the winds by speed without the diurnal 
breakdown, and re-calculated the statistics. This addendum provides the results of the updated 
wind speed and wind direction analyses. 


A.1 Diurnal Stratification 


The AMU stratified the model forecast data into two categories of 12 daytime hours (0600- 
1759 EDT) and 12 nighttime hours (1800-0559 EDT) that generally conform to day and night 
hours in the warm season months, and calculated the RMSE for wind speed and direction. 
Figure 20 shows the wind speed (ms 1 ) RMSE versus model forecast hour: “All” values are in 
blue, “Day” values are in red and “Night” values are in green. The values for All are the same as 
those in Figure 5 from the initial analysis. Figure 21 is the same as Figure 20 but for wind 
direction (degrees). As with wind speed, the values for All are the same as those in Figure 6 
from the initial analysis. For each forecast hour, the RMSE values for all times and both diurnal 
stratifications in wind speed and direction are about the same. This demonstrates that the model 
wind speed or wind direction forecasts for the 2014 warm season were not influenced by diurnal 
effects. 
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Figure 20. Wind speed (ms 1 ) RMSE versus model forecast hour 
stratified diurnally. All values are in blue, Day values (0600-1759 
EDT) are in red and Night values (1800-0559 EDT) are in green. 
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Figure 21 . Same as Figure 20 but for wind direction (degrees). 

A.2 Speed Stratification 

Since the diurnal results did not give insight into the model forecast wind direction behavior, 
the AMU stratified the data by wind speed categories to determine if this would show differences 
in the wind direction RMSEs. Three categories were selected: “<5 ms" 1 ” is considered light and 
variable, “5-10 ms" 1 ” is considered moderate and “>10 ms" 1 ” is considered strong winds. Table 2 
shows the wind speed categories used and their associated sample sizes for each forecast 
hour. Note that the sample sizes for the >10 category are below 10 regardless of forecast hour, 
too small to calculate meaningful statistics. Therefore, this category was not used for the model 
forecast wind direction verification. The other two categories, however, have enough samples 
with which to calculate a robust RMSE. 


Table 2. List of model forecast wind speed sample sizes for each category per model forecast hour. 

Wind 

Category 

Forecast Hour 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

<5 ms' 1 

2966 

2925 

2852 

2807 

2833 

2861 

2757 

2661 

2538 

2390 

2286 

2150 

2004 

5-10 

ms' 1 

405 

443 

512 

552 

528 

497 

462 

416 

404 

403 

373 

366 

372 

>10 ms' 1 

0 

0 

2 

5 

4 

4 

5 

7 

3 

5 

5 

6 

4 


Figure 22 shows the wind speed (ms" 1 ) RMSE versus forecast hour for each wind category: 
“All Speeds” is in purple, “<5” is in blue and “5-10” is in red. The values for All Speeds are the 
same as those from the initial analysis. Regardless of forecast hour, the RMSE remains <2 for 
all three categories. The values for the 5-10 category are slightly higher than the other two 
categories. Nonetheless, these are small differences between the forecast and observations 
considering the normal magnitude of the parameter. Figure 23 is the same as Figure 22 but for 
wind direction where, again, the values for All Speeds are the same as those from the initial 
analysis. This chart clearly shows the wind direction RMSE from the All Speeds category is 
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highly influenced by the <5 category. The light winds in this category tend to have more 
variability in their direction that can result in greater forecast error. The higher winds in the 5-10 
category tend to have a more stable direction and are therefore easier to forecast. The RMSE 
for this moderate wind category ranged from 15-40 degrees, which is about 20 degrees less 
than the RMSEs of the All Speeds and <5 categories. 
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Figure 22. Wind speed (ms 1 ) RMSE versus model forecast hour 
stratified by wind speed. The All Speeds category is in purple, <5 is 
in blue and 5-10 is in red. 
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Figure 23. Same as Figure 22 but for wind direction (degrees). 

A.3 Continuing and Future Work 

The AMU will continue to generate verification statistics for the 1.33-km WRF-EMS domain 
as data become available in future cool and warm seasons. As a potential future task, the AMU 
could conduct a more in-depth literature review and recalculate verification statistics once a 
larger database has been acquired. More data will produce more robust statistics and reveal a 
more thorough assessment of model performance. 
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Mention of a copyrighted, trademarked or proprietary product, service, or document does not 
constitute endorsement thereof by the author, ENSCO Inc., the AMU, the National Aeronautics 
and Space Administration, or the United States Government. Any such mention is solely for the 
purpose of fully informing the reader of the resources used to conduct the work reported herein. 
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